Laser radar (LADAR) is a rapidly evolving photonic technology with significance for both military and commercial users. Laser radar is still in its infancy with considerable R&D resources being directed at the technology. LADAR commands this interest due to the significant performance benefits it offers such as greater spatial and range resolution when compared to conventional radar and sonar. The extension of LADAR technology to provide imaging in addition to range information rather than an either/or situation has opened a host of new applications. Imaging laser radars have already found applications in both government defense and commercial business areas. There are several companies that offer commercial products that use range imaging for a variety of applications. These products will be briefly discussed since they demonstrate the usefulness of range imaging systems and simultaneously show the limitations of existing range imaging systems.
The first product to be discussed is LASAR from Perceptron Corp. in Farmington Hills, Mich. the LASAR camera dramatically expands the potential of machine vision by providing both a 3-D range image and the normal 2-D intensity image of surfaces, objects and scenes. An example of its capability is illustrated with reference to FIGS. 1a and 1b, which were taken from Perceptron's LASAR product brochure. The vase of flowers is barely discernible in the 2-D intensity image of FIG. 1a; but in the 3-D image of FIG. 1b, the vase of flowers is clearly visible. The 3-D information is displayed on 2-D displays by a variety of techniques. One technique is to display the range of each pixel as a color, and the result is called a "false color" image illustrated in FIG. 2. Another is to only display pixels that have range values between a minimum and maximum "gate" while all others are displayed black. This "range gate" method is used to produce the display shown in FIG. 3. The LASAR camera is a time-of-flight, CW (tone), amplitude modulated laser radar, which is sequentially raster-scanned over the scene within the camera's field-of-view (FOV), making a range measurement at each pixel location. The typical time to produce an image like the one shown in FIG. 1b is three to seven seconds. This product requires an accurate two-axis scanning (beam-steering) system. Since the LASAR is intended for robotic vision in factory automation applications that are typically short-ranged situations, the optical components are small, and two-axis scanning is much less of a problem as compared to long-range applications where larger optical components would be required;. The larger optical components would have greater inertia and force the two-axis scanning system to be larger and slower, making frame times even longer. The round trip time of the laser light, from the laser to the target and back to the receiver, is not a factor for the LASAR application; but in longer range applications, it could be the limiting factor in the frame rate.
The next range imaging product to be discussed is produced by Schwartz Electro-Optics, Inc. (SEO), and is called Treesense, and has been described in U.S. Pat. No. 5,275,423 to Wangler, et al. Treesense is a sensor used in the agricultural industry for controlling the spray pattern of large spray machines. The Treesense is a pulsed, time-of-flight range measuring system which has separate transmit and receive apertures that are continuously scanned in a vertical plane by a rotating mirror. This vertical scan plane is perpendicular to the axis of the forward motion of the spray vehicle. The position of the mirror, and correspondingly the laser beam, is determined by a shaft encoder attached to the motor driving the mirror. The sensor detects the presence and profiles of trees within a 90.degree. sector on each side of the spray vehicle. The presence or absence of foliage in the various angular zones is used to control the spray nozzles for each of the appropriate zones. This type of arrangement is identical to the military "push-broom" sensors. A "false color" image from the Treesense is shown in FIG. 2 (note that the axis in the direction of the tree row is obtained by a "distancetraveled" sensor mounted on a wheel of the spray vehicle). In the false color image, range is represented by the various colors. The laser in the Treesense is pulsed at 90 kHz when the mirror rotates at 40 rev/sec. Each revolution produces one scan on the left side of the spray vehicle and one scan on the right side. The Treesense is a special type of range camera that has only a one-axis scanner and uses the vehicle's forward motion to obtain the second axis of the frame.
The two Laser Radar Imaging systems cited above are special situations that, by their particular application, minimize many of the problems of more general laser radar imaging systems which will be described with reference to FIG. 3. The normal video (TV) system stares at the scene and receives all the energy backscattered from the scene that is in the receiver's FOV. The video system is fairly simple in that the receiver can be a small Charge Coupled Device (CCD) array and the system is passive. This means that the source of illumination is the solar background illumination, so there is no need for a laser transmitter. The CCD array comprises many photodiodes in a two-dimensional array that is located in the focal plane of the camera's lens, such that each of the elements in the array has a small FOV (beamlet), and the sum of all element FOVs makes up the overall camera FOV. The video system gives excellent x-axis and y-axis resolution, but there is no z-axis resolution since the CCD element integrates all the power coming from the scene that is contained within that element's beamlet. The power in the element FOV is integrated over a time which starts just after the element is read on one frame until the time it is read on the next frame. The major problems with normal TV systems are that they do not work in complete darkness and they have no resolution in the z-axis. Night-time operation requirements led to related technology areas, which are low light video (L.sup.2 V) or Image Intensified video (I.sup.2 V) and Forward Looking Infra Red (FLIR) systems. The I.sup.2 V systems are the night vision devices used by government facilities around the world. Night goggles and night vision sights are also used by civil law enforcement agencies of many countries. The use of FLIR systems is much more limited due to high cost and technical sophistication. The I.sup.2 V system is essentially a CCD camera with a photon amplifier situated in the focal plane of the camera's lens which maps the scene from the focal plane to the CCD. Night vision devices normally cannot work in daylight because the brighter scene illumination is many orders of magnitude greater than the illumination at night and this high input level causes damage to the image intensifier.
The FLIR system senses "black body" radiation coming from the scene. Every object at a temperature above 0.degree.K., or -273.degree. C., emits black body radiation. This black body radiation is electromagnetic radiation, the same as normal light, laser, radio or microwave radiation, and all are governed by the same physical laws. A difference between a normal video system and a FLIR system is the wavelength region at which its detector is sensitive. When considering spectral irradiance from an object at temperatures between 0.degree. C. and 100.degree. C., it is observed that the power at .lambda.=10 .mu.m is approximately three orders of magnitude greater than at .lambda.=2 .mu.m. This is a reason why most FLIR systems operate in the wavelength region around 10 .mu.m. This causes a major problem in the detector area since the energy of a photon at 10 .mu.m is only 0.124 a electron volts. The photo detectors used in the visual region of the spectrum are normally semiconductors. Silicon is used in the CCD devices. Silicon has an energy gap of 1.106 electron volts between the valence band electrons and the conduction band electrons in the material. A photon entering a silicon detector imparts its energy to an electron and causes the electron to move from the valence band to the conduction band. The energy of a photon is related to its wavelength by e.sub.p =1.24/.lambda.(.mu.) in electron volts. The exchange of energy between a valence band electron is an all-or-nothing exchange and is not cumulative. The energy gaps for germanium and silicon semiconductors are 0.67 and 1.106 electron volts respectively. Silicon photo detectors have a peak response at .lambda.=0.8 .mu.m, and at 1.04 .mu.m the quantum efficiency (ration of photons to electrons produced) is 1/3 that for .lambda.=0.8, even though the photon energy is greater than the energy gap. This means that a 10 .mu.m detector must have material with very low energy gaps between the valence and conduction bands. There are materials that meet this requirement, but the energy of thermally excited electrons also have this much energy. This means that 10 .mu.m detectors need to be cryogenically cooled. Due to many problems, initially there were no 2-D array detectors in FLIR systems. The FLIR systems either used a single element that was serially raster scanned over the entire FOV or linear arrays of detectors that were scanned in one axis. Many FLIR systems were generally made from common standardized modules. The common module receiver was a linear array of 120 elements in which each element had its own energy receiver. Therefore, a FLIR system that had 400 horizontal and 240 vertical pixels required two common FLIR receiver modules (120 pixels each) which were scanned horizontally thirty times a second in order to be compatible with normal TV monitors.
The common module FLIR system requires one axis of scanning, although there has been much effort to develop staring focal plan array FLIR systems. The advantages of the staring focal plane FLIR system are that no scanning would be required and the energy would be larger since the black body power radiated toward the FLIR could be integrated for the entire frame period rather than the frame period times. (n.sub.h).sup.-1, where n.sub.h is the number of pixels in a horizontal line.
There is a misconception that if one had an n.times.n photodetector that an n.times.n Laser Range Imaging system would be just a minor design problem. There are major differences between a passive 2-D imaging system and an active 3-D laser range imaging system. It is true that a 2-D focal plane detector is required, but such detectors have been available for some time. In fact, one such detector was featured on the cover of the March 1988 issue of Laser & Photronics Magazine by Gordon Publications, Inc., Dover, N.J. 07801-0952. This was a 464 element, silicon photodiode made by Centronics, Inc. There have been multi-element 1-D and 2-D APD arrays made by at least two vendors (Advanced Photonics, Inc. and EG&G). Advanced Photonics, Inc. (API) and SEO have worked closely on other programs, and API will produce a developmental APD array for this Project.
Again with reference to FIG. 3, if a CCD Camera was used in place of the Laser Radar, the number of pixels in X (azimuth) and Y (elevation) axis would typically be in the range of 200 to 500. The pixel element of CCD Array Camera would receive all optical energy scattered by an object which is anywhere within pixel FOV along X axis and Y axis, and at any range from zero to infinity. Thus, the CCD camera gives very good angular resolution but does not have any range or distance resolution. By using two CCD cameras with collinear optical axes and a known baseline separation, the range of an object can be determined from range and angle data as recorded by the two cameras. By doing this for all distinguishable features common to both camera's field-of-views (FOV) a 3-D image could be produced. This system has the advantage of being passive (no scene illumination is required) if the scene is visible to the human eye. However, if the scene was completely uniform there would be no contrast points to compute range from the angle change as viewed by the two cameras. An example of this would be rolling hills covered with a wheat crop. When there are plenty of contrast points, the process of determining surfaces are computationally intensive. Another disadvantage is that the system needs scene illumination. In daylight conditions this is not a problem, but if night operation is required this is a problem.
Active systems for measuring range have been in existence for many years. These range measuring devices utilize nearly all regions of the electromagnetic spectrum from low frequency radio waves to light wave lengths approaching x-rays. Range measuring systems have also used other technologies other than electro-magnetic devices (laser and microwaves), an example of this is the ultrasonic range measuring used in the medical field (sonograms).
The subject of the present invention is a range imaging system obtained by an active optical system. A scene will intersect the imaged/ranged space at various pixels contained in the total solid FOV of the range imaging system. The coordinates where the scene intersects the pixels is the measurement that the imaging radar must make. There are several different approaches currently used to produced a range imaging system, i.e., which satisfy requirements of measuring the X (azimuth), Y (elevation) and Z (range) of each pixel of scene intercept within the sensor FOV. One approach used is to sequentially raster scan a range finder whose transmit beam and receiver FOV are the same size as the desired pixel resolution. Such a system would require scanning the transmitter beam and receive FOV in two axes. Such a system would start in the upper left corner of the imaged/ranged space shown in FIG. 3. At this location the laser transmitter would transmit a signal toward the scene and the receiver would measure the range and intensity of the reflected return signal. After completing the measurements, the device is then pointed at the next pixel to the right and the measurement process is repeated, etc. After making measurements of all pixels in the top row of pixels, the steering mechanism then points the range finder to the first pixel in the second row and measurements are made for each pixel in that row. This process continues until the entire imaged/ranged space of FIG. 3 is covered. In order to have the range images do what is comparable to normal video camera images, the number of pixels must at least 128 X-pixels by 128 Y-pixels or a total of 128.sup.2 (16,384) pixels. This sequential raster scanning system requires a 2-axis scanning device which is required to point the laser range finder to 16,384 different positions for each frame. This type of range imaging is only suitable for short range systems, Long range systems require a large receive aperture in order to collect enough reflected signal to make the range measurement. Also, it is generally the practice to have the receiver FOV the same size as the laser beam in order to minimize the background noise. However this further complicates the scanning process. When scanning the point in the scene that the laser transmitter will illuminate is determined at time of transmission and the required direction lag of the receiver FOV to receive the reflected signal from this point is determined by the round trip light propagation time. In other words, if the transmit beam and the receive FOV were the same size, the scanning system would be have to be step raster scanned. Even ignoring, the scanning problem (assuming no time required to scan from one position to the next position), the number of frames per second is limited simply due to the round trip light propagation time. For example, a scene at 2 kilometers would have 13.3.times.10.sup.-6 round trip propagation time so that a 128 by 128 sequential scan system would take 16,384.times.13.3.times.10.sup.-6 or 0.22 sec to do a complete scan without allowing any time for making measurements. So it seems that even without time allotted for scanning, data measuring, or data processing time, a 2 km sequentially scanned system would be limited to 4.5 frames per second which is a factor of 10 less than a normal video system. The major disadvantage of a 2 km sequential raster scan is that 2-axis scanning is limited to low frame rate and requires step scanning of an enlarged receiver FOV.
Another type of active range imaging system has a laser transmitter and gated image intensifier CCD Array as a receiver. The receiver is a normal video camera with a photo multiplying micro-channel plate in front of the CCD array camera detector. Here the only signal of interest is the reflected laser energy. A narrow band width optical filter, whose band pass center wavelength is the wavelength of the laser, is placed in front of the receiver aperture or in an optical path between aperture and micro-channel plate photo multiplier. The purpose of the filter is to eliminate as much as possible all light coming from the scene except the reflected laser energy. Unfortunately, the solar energy has energy in the spectral region beginning in the UV and extending to wavelengths beyond the far infra red spectra region. This peak energy region is in the visual region (400 to 700 nm wavelengths). These physical laws and physical constraints establish how spectrally pure the laser can be and how narrow the optical bandpass filter can be in a practical system. Consider the restrictions a practical system might utilize with a 1 nm wide optical band pass filter when a solid state laser (such as Nd:YAG, Nd:YVO.sub.4 or Nd:YLF) is used as the laser transmitter. It is reasonable to use a 10 Angstrom or 0.1 nm bandpass filter. This type system uses a gated micro channel plate image intensifier for both the gain it provides for the small amount of energy passing through the narrow bandpass filter and also to shutter the receiver in order to determine range. The range determination operates by firing the laser and at a precise time later, opening the shutter (gate the micro channel plate (MCP) on). If the scene surface is within the FOV at range R=T.sub.1 C/2 (where T.sub.1 =time interval and C=speed of light) there will be reflected laser from those pixels. The MCP gating needs to be narrow in order to prevent pixels at greater range to be present also. The range resolution of the system is the time convolution of the gating and the laser pulse wave form. Referring back to FIG. 3, this system looks at all X and Y pixels at once but only one Z axis pixel. The gating or shuttering signal range gates laser reflected energy, but does not range gate background reflected energy i.e, it time gates or time limits the acceptance of the solar background reflected energy. A simple example will illustrate the importance of the statement. A small aircraft occupies one-half the cross section area (x,y) of one pixel at some range R/C and the rest of the FOV is filled with bright white cloud background. All the CCD pixels will have the same energy recorded for gating times except the pixel containing the aircraft. That pixel will have the same output for all Z gate times except the gate time corresponding to the range of the aircraft. The energy for this gate time is the sum of solar background reflected from the bright white cloud behind the aircraft (1/2) pixel during the gate time and aircraft reflected solar background plus the laser reflected energy from the aircraft. This example shows the gate time limits the solar background but does not limit receiving background from objects located at ranges corresponding to the gating time.
In the CCD camera the received laser energy during the range gate time must be greater than the received background energy in the adjacent pixels by a factor of 2 in order that the pixels with target can be distinguished from the pixels without target. In the range finder case, the instantaneous light intensity must be detected so that range can be measured. In order to detect the laser pulse, the reflected laser pulse must produce a signal that is at least four times greater than the root mean squared "shot" noise caused by the background solar light. Therefore, for a range finder silicon detector, the receive power must produce a signal current which is four times the rms noise current. When comparing the required power for the gated CCD and the range finder detector, the CCD minimum detectable power is 100,000 times greater than that of the range finder detector. This assumes that the dominate noise in both systems is the background which may not be the case. Typically, each system must be analyzed on an individual basis. Well-designed range finders have been made which have had minimum detectable signal in the 5 to 10 photons region which is significantly less than that required for a shuttered CCD system.
The sequentially range gated CCD receiver-laser flood light illuminated scene system has the advantage of no scanning required. However, it has the disadvantage of low sensitivity in high background, many range cells to search for the scene if short gate times are used for good range resolution, a very-high power laser is required since the laser energy is spread over the entire scene and finally poor efficiency of the laser power since the shutter only receives scene pixels which are at ranges which corresponds to the shutter time.